There are various applications in which large amounts of data generated in computing environments are pushed to one or more servers in a cluster server for real-time processing. Such applications include, for example, sensor based monitoring (e.g., network of Internet of Things sensors for industry monitoring), financial anti-fraud monitoring, stock trading, web traffic monitoring, network anomaly monitoring, machine learning (ML), deep learning (DL), big data analytics, or other high-performance computing (HPC) applications, etc. These applications generate a continuous stream of records (or events), which can be pushed to a distributed computing system (e.g., distributed stream processing system) that is configured for large scale, real time data processing and analysis of such data streams. A distributed computing system comprises a large scale of shared computing resources that are distributed over a cluster of computing nodes. Techniques for implementing an efficient distributed computing environment for data stream analytics and HPC applications is not trivial as the intensive computational workloads, and the massive volume of data that must be communicated, streamed, prefetched, checkpointed, and coordinated between the shared computing resources of the distributed computing system presents a significant challenge and practical limit on system performance and scalability.